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ABSTRACT 

E)ecision  makers  are  often  confronted  with  problems  for  which  there 
exist  several  distinct  measures  of  success.   Such  problems  can  often 
be  expressed  in  terms  of  linear  or  nonlinear  programming  models  with 
several  "criterion"  functions  instead  of  single  objective  functions. 
A  variety  of  techniques  have  been  applied  to  multicriterion  problems, 
but  the  approach  used  here,  "The  Dynamic  Programming  Approach  to  Multi- 
criterion  Optimization  Problem,"  is  based  on  the  concept  that  the  ideal 
solution  to  a  multiobjective  problem  must  be  a  pare to  optimal  solution. 
In  many  cases  simply  narrowing  the  set  of  candidate  solutions  to  the 
set  of  all  pareto  optimal  solutions  may  enable  the  decision  maker  to 
find  the  compromise  being  sought.   The  determination  of  nondominated 
points  and  corresponding  nondominated  values  (pareto  optimal  solution) 
related  to  the  multicriterion  optimization  problem  is  approached  through 
the  use  of  dynamic  programming.   The  dynamic  programming  approach  has  an 
attractive  property  which  provides  the  basis  for  generation  of  non- 
dominated  solutions  at  each  stage  by  the  decomposition  method.   By 
using  recursive  equations  we  can  find  out  the  nondominated  points  and 
corresponding  nondominated  solutions  of  multiaggregate  return  f\inction. 
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I.   INTRODUCTION 

A.   PROLOGUE 

Situations  requiring  a  decision  typically  involve  multiple  conflict- 
ing goals  which  cannot  be  satisfied  simultaneously  due  either  to  the 
inherent  conflict  among  some  goals  (e.g.,  maximizing  comforts  in 
traveling  between  two  points  vs.  minimizing  cost  of  travel-   Maximizing 
effectiveness  vs.  minimizing  cost)  or  to  a  scarcity  of  resource  (e.g., 
maximizing  productions  on  each  of  several  individual  production  lines) . 

Since  first  described  by  Pare to  many  years  ago,  the  problem  of 
multicriteria,  or  vector  valued  optimization  has  fascinated  economists, 
planners  and  engineers.   Although  it  has  also  been  considered  in  a  very 
abstract  framework  by  a  number  of  researchers  the  most  commonly  en- 
countered form  of  the  problem  seems  to  be  as  follows: 

Maximize  [f^(x),  f2(x),...,  f^^^)  ] 
subject  to  g.  (x)  _^  0,  i-1,2  , .  .  .  ,m 

where  x  is  a  vector. 

During  the  past  10  years  increased  research  in  this  area  has  been 
directed  in  several  directions  reflecting  different  classes  of  multiple 
objective  decision  problems.   An  excellent  recent  summary  of  this  area 
is  by  K.  R.  MacCrimmon,  [14].   Roy  [15]  summarized  several  approaches  in 
1971  and  Johnsen  [16]  offers  an  almost  encyclopedic  coverage  of  the 
field  to  1975. 


A  lot  of  researchers  solved  the  MCO  problem  by  using  the  concept  of 
a  nondominated  solution.   Yu  [l7]  introduced  a  general  concept  of  domi- 
nation structures;  the  idea  helps  to  view  the  one-dimensional  comparison 
and  the  set  of  nondominated  (pareto  optimal)  solutions  as  two  special 
cases  of  a  more  general  solution  concept.   Zeleny  [is]  develops  a  multi- 
criteria  simplex  method  for  generating  all  nondominated  extreme  points. 
Polak  [19]  presents  an  algorithm  which  constructs  an  economical  grid  of 
noninferior  points  to  be  used  in  computation  with  an  interpolation 
scheme  in  the  value  space.   The  algorithm  is  specifically  designed  for 
the  bicriteria  case,  but  it  can  also  be  used  in  higher  dimensional 
situations . 

The  algorithm  we  shall  present  in  this  thesis  is  specialized  to  the 
two  or  more  criteria  space  with  only  a  few  equality  or  inequality  type 
constraints  (only  one  constraint  in  our  example) . 

The  theory  of  the  dynamic  programming  approach  to  the  multiple  cri- 
teria optimization  problem  [mcodp  ]  is  developed  by  combining  the  non- 
dominated  solution  concepts  and  dynamic  programming  method. 

Using  the  dynamic  programming  decomposition  theorem,  we  break  the 
multicriteria  problem  into  N  stages .   At  each  stage  we  find  the  non- 
dominated  solution  by  a  recursion  equation  and  hence  solve  the  MCO 
problem  by  finding  the  nondominated  solutions  of  multi-aggregate  return 
function. 


B.   MULTI CRITERION  OPTIMIZATION  PROBLEM  ANALYSIS  - 
A  REVIEW  OF  THE  LITERATURE 


Maximize  F(x) 


whe 


siibject  to  g.  (x)  <  0   for  i=l,2,...,m 
re  X  is  an  N-vector  (x^ ,  x_,...,x  ),  and  F(x),  a  function  of  x,  is 


an  L- vector  with  scalar  components  (f  (x) ,  f  (x) , . . . ,  f  (x) ) .   The 

J.  Z  Li 

solution  to  this  problem  is  the  set  of  all  x  which  are  nondominated  and 
which  satisfy  the  constraints  g. (x)  <  0  for  i=l,2,...,m. 

A  solution  vector  x  is  nondominated  if  it  satisfied  the  constraints 
and  if  there  does  not  exist  another  x'  also  satisfying  them  and  having 
the  properties  that 

f.(x')  <  f . (x)   for  i=l,2,...,L 
X      =   1 

f . (x')  >  f . (x)   for  at  least  one  such  i. 
1        1 

Although  interest  in  multicriterion  problems  is  quite  recent,  the  liter- 
ature concerning  these  problems  is  already  voluminous. 

For  a  bibliography  and  synthesis  of  work  in  this  area  see 
MacCrimmon  [2]  .   In  his  paper,  written  in  1973,  he  discussed  every 
method  which  was  developed  up  to  then  and  attempted  to  group  them  into 
four  main  categories  to  facilitate  comparisons  among  the  various  methods , 
They  are 

a)  weighting  methods 

b)  sequential  elimination  methods 

c)  mathematical  programming  methods 

d)  spatial  proximity  methods. 


The  class  of  weighting  methods  has  received  the  most  attention  and 
particular  models  within  this  class  have  been  the  most  widely  applied. 
Although  these  weighting  methods  seem  very  diverse,  they  all  have  the 
following  characteristics: 

-  A  process  comparing  attributes  by  obtaining  numerical  scalings  of 
attribute  values  (intra-attribute  preference)  and  numerical 
weights  across  attributes  (inter-attribute  preference) . 

MacCrimmon  divided  the  weighting  method  into  nine  weighting  methods. 
Some  of  them  are: 

linear  regression 
trade-offs 

simple  additive  weighting. 
The  linear  regression  method  can  be  used  when  the  problem  is  in  such 
situations  that  are  sufficiently  repetitive  to  group  together.   Attri- 
butes considered  by  the  decision  maker  can  be  treated  as  variables  in 
a  linear  model.   From  his  past  choice,  coefficients  of  the  attributes 
are  estimated  using  standard  regression  techniques. 

Slovic  and  S.  Lichtenstein  [3]  provide  a  very  comprehensive  survey 
of  linear  regression  models  and  have  an  excellent  bibliography. 

The  trade-off  methods  differ  from  the  linear-regression  method  in 
that  the  preference  of  the  decision  maker  is  obtained  by  directly  asking 
him  his  preferences  rather  than  by  inferring  them  from  his  past  choices . 
Although  this  method  has  the  advantage  of  obviating  the  need  for  a  con- 
siderable past  history  of  similar  situations,  it  has  the  disadvantage 
of  possibly  finding  that  the  decision  maker  is  unable  to  verbalize  his 
true  preferences. 


In  the  trade-off  approach,  the  marginal  rates  at  which  the  decision 
maker  is  willing  to  trade  one  attribute  for  another  are  obtained  by 
direct  questioning.   If  all  the  relevant  trade-offs  can  be  made,  then 
the  focus  of  the  problem  can  be  narrowed  to  one  attribute.   Raiffa  [5] 
utilizes  a  more  general  trade-off  method  on  the  very  complex  problem  of 
obtaining  certainty  equivalents  for  lotteries  in  multiple  attribute 
decision  problems. 

In  the  simple  additive  weighting  method,  for  each  of  the  attributes 
decision  problems. 

In  the  simple  additive  weighting  method,  for  each  of  the  attributes 
the  decision  maker  assigns  importance  weights  which  become  the  coef- 
ficients of  the  variables.   To  reflect  his  marginal  worth  assessments 
within  attributes,  the  decision  maker  also  makes  a  numerical  scaling  of 
intra-attribute  values.   He  can  then  obtain  a  total  score  for  each 
alternative  simply  by  multiplying  the  scale  rating  for  each  attribute 
value  by  the  importance  weight  assigned  to  the  attribute  and  then  sxamming 
these  products  over  all  attributes . 

After  the  total  scores  are  computed  for  each  alternative,  the  alter- 
native with  the  highest  score  is  the  one  prescribed  to  the  decision 
maker.   Simple  additive  weighting  is  very  widely  used.   One  important 
use  of  this  technique  was  in  the  mustering  out  system  of  the  U.  S.  Airmy 
at  the  end  of  World  War  II  [6]. 

Sequential  elimination  methods  are  less  demanding  of  the  decision 
maker  than  weighting  methods.   Sequential  elimination  methods  are 
characterized  by  a  process  for  sequentially  comparing  alternatives  on 
the  basis  of  attribute  values  so  that  alternatives  can  be  either  elimi- 
nated or  retained. 
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MacCrinimon  distinguished  among  four  methods .   One  of  these  is  the 
dominance  method  which  compares  one  real  alternative  against  another 
real  one  to  see  if  proper  alternatives  can  be  eliminated.   If  one 
alternative  has  attribute  values  that  are  at  least  as  good  as  those 
of  another  alternative  for  all  attributes,  and  if  it  has  one  or  more 
values  that  are  better,  then  the  first  alternative  is  said  to  "dominate" 
the  second,  and  the  second  alternative  can  be  eliminated.   Although  this 
is  probably  the  least  controversial  decision  rule,  it  unfortxmately  does 
not  often  succeed  in  eliminating  very  many  alternatives . 

Terry  [7]  shows  how  dominance  can  be  used  as  an  initial  filter  in 
choosing  among  new  product  areas  for  comparing  diversification.   Freiner 
and  Yu  [8]  combine  dominance  with  the  mathematical  programming  approaches 

The  class  of  programming  methods  has  recently  begun  to  receive  much 
attention.   MacCrimmon  divided  the  mathematical  programming  methods  into 
three  methods,  linear  programming,  goal  programming  and  inteiractive , 
multi-criterion  programming. 

A  regular  linear  programming  model  may  be  viewed  as  a  multiple 
attribute  decision  method.   The  variables  are  the  attributes.   The 
linear  constraints  are  conjvinctive  constraints  on  combinations  of 
attributes,  and  there  is  a  linear,  compensatory  objective  function. 

Efficient  solution  algorithms  are  available.   The  classic  diet 
problem  [9]  is  an  example  of  this  method.   The  objective  is  to  provide 
a  balanced  diet,  through  the  selection  of  the  amounts  of  particular 
foods,  that  will  satisfy  particular  nutritional  standards  at  minimum 
cost. 

In  a  goal  programming  foinnulation,  the  decision  maker  specifies 
acceptable  or  desired  levels  on  single  attribute  values  (i.e.,  one- 
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variable  constraints)  or  on  combinations  of  attributes  (i.e.,  multi- 
variable  constraints)  and  these  serve  as  the  primary  goals. 

Lee  and  Claytor  [10]  apply  goal  programming  to  the  scheduling  of  an 
academic  department.   The  attributes  (variables)  are  defined  in  terms 
of  faculty  and  non- faculty  personnel  having  varying  degree  qualifica- 
tions.  The  goals  involve  faculty /student  ratios,  faculty/staff  ratios, 
percentage  increase  in  salaries,  low  cost,  etc.   Solutions  obtained 
lander  different  assimiptions  about  available  budget  and  accreditation 
requirements  are  presented. 

Ijiri  [11]  develops  the  accounting  aspects  of  goal  programming. 

Interactive,  multi-criterion  programming  does  not  ass\mie  a  global 
objective  but  rather  requires  the  decision  maker  to  provide  his  local 
trade-offs  in  the  neighborhood  of  a  feasible  alternative.   These  trade- 
offs (on  the  attributes  or  criterion  involving  attributes)  are  used  in 
a  local  objective  function  for  a  mathematical  programming  algorithm  to 
generate  an  optimal  solution  for  that  objective.   The  decision  maker 
then  has  an  opportunity  to  provide  new  trade-offs  which  again  serve  as 
inputs  to  the  algorithm.   This  process  continues  until  the  decision 
maker  no  longer  wishes  to  revise  his  trade-offs  and  so  an  optimal 
solution  is  reached. 

Geoffrion,  Dyer,  and  Feinberg  [12]  describe  the  use  of  the  method 
in  scheduling  an  academic  department.   Related  approaches  are  given  in 
the  studies  of  Benayoxm  de  Montgolfier  and  Tergny  [13]  and  Yu,  Zeleny. 

While  the  preceding  three  classes  are  probably  the  most  common 
general  approaches  to  miiLti-attribute  decision  problems,  some  of  the 
more  specialized  methods  are  also  receiving  attention.   Since  they  all 
make  explicit  use  of  spatial  representation,  MacCrimmon  calls  them 
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"spatial  proximity  methods."   He  distinguished  among  three  methods  in 
spatial  proximity  methods.   One  of  them  is  the  indfiference  map  method. 

The  decision  maker's  preferences  can  be  obtained  in  the  form  of 
indifference  surfaces  which  show  the  combinations  of  attribute  values 
that  are  equally  preferred.   The  alternatives  to  be  considered  can  be 
located  in  the  same  spatial  representation,  and  by  identifying  the 
indifference  surface  on  which  they  lie,  a  complete  ordering  among  the 
alternatives  can  be  generated. 

MacCrimmon  [14]  gives  an  example  of  the  use  of  this  procedure  for 
transportation  system  planning.   Many  combined  approaches  are  possible 
and  hopefully  this  overview  will  provide  some  insight  into  the  available 
methods  so  that  readers  can  begin  to  consider  useful  combinations  for 
their  own  multiple  objective  decision  problems. 
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II.   BASIC  THEORY  OF  DYNAMIC  PROGRAMMING  APPROACH 
TO  MULTICRITERION  OPTIMIZATION  PROBLEM 

A.   INTRODUCTION 

This  chapter  is  concerned  with  finding  solutions  to  the  following 
general  problem,  which  will  be  called  "MCO  (Multicriterion  Optimization) " 
problem. 

Find  the  set  of  nondominated  solutions  to 


maximize      f  (x) 


fj_(x) 


subject  to 

Ax  <  b 
X  >  0 

where  A  is  an  M  by  N  matrix 


b  is  a  vector  (b-,b^,...,b  ) 
12       N 


X 


is  a  vector  (x  ,x  , ...,x_) 


f .  (x)  is  a  decomposable  f\anction 

The  dynamic  programming  approach  to  the  multiple  objective  function 
problem  is  of  interest  because  we  can  decompose  each  objective  fxonction 
into  N-stages  in  dynamic  programming  (if  separability  and  monotonicity 
conditions  hold).   At  each  stage,  we  get  the  L  single-variable  return 
fxinctions  and  we  can  find  the  nondominated  policy  by  just  eliminating 
the  dominated  policy. 
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Once  we  get  the  optimal  returns  (nondominated  solution)  from  each 
stage,  then  we  can  easily  obtain  a  nondominated  solution  for  a  system 
by  Bellman's  "principle  of  optimality"  stated  more  succinctly  in  his 
words  - 

An  optimal  policy  has  the  property  that  whatever 
the  initial  state  and  decision  are,  the  remaining 
decisions  must  constitute  an  optimal  policy  with 
regard  to  the  state  resulting  from  the  first 
decision  [20] . 

B .   NOTATION 

The  following  notation  and  basic  definitions  will  be  employed  in  the 
remainder  of  this  chapter. 

1.   Multistage  Problem  Solving 

When  solving  a  complex  problem,  we  often  break  it  into  a  series 
of  smaller  problems  -  decomposition  -  and  then  combine  the  results  from 
the  solution  of  the  whole  problem  -  composition.  We  call  this  approach 
multistage  problem  solving. 


n 


n 


D 


1 


Stage 

irtr 


n-l 


n 


n-l 


1 


n-l 


ITTT 


'n-l 


n-2 


\v 


i- 


D, 


1 


*  I    I 


TTTT 


[Figure   1 ] 
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2.  State  (x) 

The  state  of  each  stage  is  defined  by  a  set  of  numbers  called 
state  variables 

We  use  a  subscript  n  to  denote  the  value  of  the  state  variables  after 
stage  n  -  see  (Fig.  1) . 

3.  Decision  Variable  (D) 


The  decision  variables  are  those  quantities  that  can  be  con- 
trolled or  chosen  in  the  design  and  operation  of  any  stage 

D  =  (D  ,  D    ,  ...,    D  ,  ..  .,  D  ) 
1        Z  n       N 

See  (Fig.  1) . 

4.   Stage  Transformation 

Each  stage  of  the  process  transforms  the  state  of  its  input  into 
an  output  state  in  a  way  dependent  on  the  decision  that  has  been  made 
for  the  operation  of  the  stage.   We  express  this  symbolically  by  writing 


X  T  =  t  (X  ,  D  ) 

n-1    n  n   n 


meaning  that  given  X  and  D  it  is  possible  to  calculate  X 

5.   Multiple  Return  Function 

Each  stage  has  L  return  functions.   We  call  this  a  "multiple 
return  function"  and  the  notation  is  as  follows 
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r  = 
n 


where 


n  =  1,2,...,  N  (stages) 

i  =   1,2,...,  L  (number  of  return  functions) 


6.   Multiple  Aggregate  Return  Function 


,th 


The  total  return  R  of  the  I       return  from  stage  one  through  N 
is  some  function  of  the  I       individual  stage  returns  written  as 


R  = 


R 


R 

1_  u 


X/  X^rX/  X*  ^T 


i   =  1,2,...,  L  (number  of  return  function) 

7.   Inequality  Notation: 

If  the  objective  functions  of  an  optimization  problem  is  vector 
valued,  such  that 


—     12        £  T 

Q=[Q>Q>--'.Q], 


we  define  a  partial  ordering  of  elements  Q  as  follows: 
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—  o   —  *     oi    *i 

Q      <   Q         ^  Q         <   Q  ^^ 


—  o   —  *     oi    *i 


and  3  j  9-Q°^  ji   Q*^ 


—  o   —  *     oi    *i 

Q    IQ      ^  Q      <Q  \ 


—  r^    — * 


.01 


Q     =  Q      ^  Q       =Q     \ 


8 .   Notation  of  Nondominated  Points 


Given  a  set  of  feasible  points 


V  c  r"" 


and  a  vector  criterion  function 

n    P 
f  :  R  ->  R 

then  the  set  of  nondominated  points  is 

Q  =  {x*e  v|  f(x"'')  >  f(x*)  =  >  x"*"  i  V} 


9.   Notation  of  Nondominated  Values 


n  =  {Z  =  R^  I  z  =  f  (x)  ,  xefi} 


The  set  of  all  nondominated  solutions  in  the  criteria  space  T  (X  )  will 

N   N 

be  denoted  by 


"  [^N  %^ 


°N'  — '°1 


where  T^(y  =  {g(r^(X^,D^)  ,  r^.i(  V^.D^,^)  ,— r^CX^^D^) )  ,  ^  D^ D^} 
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that  is,  T,, (X^  )  is  the  set  of  all  feasible  values  of  the  vector 
N   N 

subject  to 

X   ,  =  t  (X  ,D  ) ,  n  =  1,2, N 

n-1    n  n  n 

andg(7^(X^,D^),  7^_^(X^_^,D^_^),  — ,  7^(X^,D^))  is  ^ 

the  multiple  aggregate  return  fvmction. 

The  optimal  return  (nondorainated  solution)  from  N  stages  is  defined  as 

°N'  — ■''l 

subject  to 


X  ,  =  t  (X  ,D  ) ,  n  =  1,2, N, 

n-1    n  n  n 


10 .  Multiple  Recursion  Equation 

Assuming  that  the  optimal  return. 


^N^\^  ^  ^  '■'^N^^^^  ^^  decomposable, 


D^.  — ,D. 


then 


where 


=n'V  =  *9i(r^(X^,D^),  F^.^(X^.^)),^.  D^} 
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that  is,    S    (X  )    is   the  set  of  all   feasible  values   of  the  vector 
^l^^N^^'V'  "  tVl^Vl^l^    for  all  values  of  D^. 

This  notation  means   that  each   r„(X^  .  D^J    is   combined  with  each  vector  in 

N      N      N 

the  set  JT  [Vl^Vl^J 

^N'  — '    °1 

siabject  to 


and  let 


Vi  =  V\-V 


2n"^N'V    -9i"^n"n'°n'-    Vi<Vi" 


then 


Applying  the   same  decomposition  to  F        (X^_   ) , ,    F    (X  ) ,   we   obtain  the 

multiple   recursion  equations. 


F    (X  )    =  fi    [S    (X   )  1    n  =   1, ,N 

n     n  •■   n     n    -" 

D 
n 


where 


S    (S   )    =    {Q    (X    ,D  )  ,   V  D   } 
n     n  ^n     n     n  n 


Q    (X   )    =   r    (X    ,D   )       n   =   1 
n     n  n     n     n 


=    r    (X    ,D   )    o   F      .(X      .,D      ,)       n   =   2, ,N 

n     n     n  n-1     n-1     n-1 


and  "o"  is  the  composition  operator 
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subject  to 


X  ,  =  t  (X  ,D  ) 
n-1    n  n  n 


C.   BASIC  CONCEPTS  OF  "NONDOMINATED" 


Let  us  briefly  outline  the  basic  concept.   We  will  utilize  the 

decision  variable  D  as  in  Figure  2 .   The  value  of  decision  variable  D 

n         ^  n 

results  in  the  value  of  Q^(X  ,D  )  through  the  equation. 


D 


Q  (X  ,D  )  =  r  (X  ,D  ) 
^n  n  n     n  n  n 


o  F    (t  (X  ,D^)) 
n-1  n  n  n 


-»■  Q   (X  ,D  ) 

*^n   n  n 


^  Q   (X  ,D  ) 

^n   n  n 


-*-  Q   (X  ,D  ) 
n   n  n 


[Figure  2 ] 


n  =  1,2,...,N  (number  of  stages) 

I   =  1,2,...,L  (nvimber  of  objective  functions) 

X  =  fixed, 
n 

The  relationship  between  decision  variables  and  return 
f  xanctions . 


Each  choice  of  decision  values  which  is  allowable  yields  a  feasible 
solution  in  criteria  space  as  in  Figure  3a. 


>'  e„  e^n'^n*' 


dominated 


[Figure  3a].   Feasible  solutions 
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The  full  set  of  allowable  solutions  obtained  by  mapping  all  allow- 
able values  of  the  decision  variable  D  into  the  criteria  space  yields 
some  volume  in  that  space,  again  as  in  Figure  3a. 

In  general,  the  majority  of  these  feasible  solutions  will  be  domi- 
nated, that  is,  a  feasible  solution  will  exist  which  is  at  least  as  good 
in  all  criteria  and  better  in  at  least  one  criterion. 

A  "nondominated"  policy  can  be  defined  in  the  following  way.   A 

point  D  *  e  X  is  said  to  be  a  nondominated  policy  if  and  only  if  there 

is  no  other  policy  D    £  X  such  that 

n     n 


0   (X  ,D  )  >  Q   (X  ,D  *) ,  for  all  i 

Ti   n  n   =  ^n   n  n 


and 


Q  ^(X  ,D  )  7^  Q  ^(X  ,D  *)  ,  at  least  one  i 
^n   n  n     ^'n   n  n 


where  X  is  fixed, 
n 


That  is,  a  "nondominated"  policy  has  the  property  that  there  is  no  other 
feasible  policy  which  would  improve  at  least  one  goal  variable  while  the 
others  stay  unchanged. 

The  surface  of  nondominated  solutions  in  criteria  space  for  the 
example  shown  is  indicated  in  Figure  3b. 


2  + 


"n 


[Figure  3b].   Surface  of  nondominated  solutions, 
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A  set  of  representative  nondominated  solutions  for  the  example  shown 
is  indicated  in  Figure  3c. 


2  \ 


o  •  • 


O   O 


O   0 


<>  0   o  o  • 


■^   1 

2n 


[Figure   3c].      Representative  nondominated  solutions. 

D.       BASIC   THEOREM 

From  the  general  M  C  O  problem, 

maximize     F(x) 

sxabject  to        g(x)    <^  0      for     i   =   l,2,...,m 

where  x  is   an  N-vector  and  F(x)    a  function  of  x,    is  an  L-vector  with 

scalar  component    (f. (x) ,    f    (x) , . . . , f    (x) ) . 

■L       ^  L 

To  use  the  dynamic  programming  method  each  of  the  L  objective 
functions  must  satisfy  the  definition  [1]  and  [2]. 

1.   Definition  [1]  (Separability) 

Let  the  aggregate  return  be 

"Separcibility"  can  be  written 

5  C^'  Vi ^1^ 

=  g^  [r^,  g^Cr^,^ r^)  ] 
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where  g.    and  g     are  real-valued  fxmctions, 


2.   Definition  [2]  (Monotonicity) 

From  g^[r^,  g^ (r^_^, . . . ,r^) ] 
g  is  a  monotonically  nondecreasing  function  of  g   for  every  fixed  r  . 
If  each  objective  function  satisfies  the  definition  [1],  [2],  then  it 
can  be  decomposed  into  N  stages.   At  each  stage  we  get  L  return 
functions . 


r  (X  ,D  )   = 
n  n  n 


r   (X  ,D  ) 

n   n  n 


r   (X  ,D  ) 

n   n  n 


r   (X  ,D  ) 
n   n  n 


n  =  1,2,...,N  (stages) 

£  =  1,2,...,L  (number  of  return  functions) 


and 


Q  (X  ,D  )  =  r  (X  ,D  )  o  F   - (t  (X  ,D  )) 
^n  n  n     n  n  n     n-1  n  n  n 


giving 


^n^\'V      = 


Q   (X  ,D  ) 
n   n  n 


Q   (X  ,D  ) 
^n   n  n 


Q   (X  ,D  ) 
*n   n  n 


We  assume  that  there  is  some  overall  utility  function  implicitly 

defined  on  the  Q  (X  ,D  ) .   At  stage  MX  is  fixed  and  the  feasible 
^n  n  n         ^     n 
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values  of  D  depend  on  X  . 
n   -^        n 


U[Q  (X  ,D  )]   =  U[Q   (X  ,D  ), 0  ^(X  ,D  )] 

•-^n  n  n  -■      *■  n   n  n     Ti   n  n  -" 


This    function  is   unknown  but  it  can  be   assumed  to  be   real-valued, 

monotonic  and  nondecreasing  in  each  argument  Q   (X  ,D  ) . 

The  problem  is  to  find  D  *  for  fixed  X  such  that 
^  n  n 


MAX   U[Q  (X  ,D  )]  =  uFq  (X  ,D  *)  ]  . 

•■^n  n  n  ■•     '■^n  n  n   ■' 
D  eX 
n  n 


3.   Theorem  [l]  [Geoffrion] 


Let  the  set  of  feasible  D  be  compact.   Then  at  least  one  policy 

at  which  UfO  (X  ,D  )  1  achieves  its  maximiam  over  X   (fixed)  is  "non- 
■•^n  n  n  -■  n 

dominated."      That  is 


MAX      U[Q    (X    ,D    )]    =   U[Q    (X    ,D    *)  ] 

D   eX 
n     n 

where  D  *eQ.   and  Q,  =   the  set  of  nondominated  points. 

PROOF 

By  the  compactness  (boiinded  and  closed)  of  the  feasible  set  of 

D  and  the  continuity  of  U  there  is  at  least  one  maximal  solution  to 

U[q_ (X  ,D  )1,  say  D  *efi.   Consider  some  point  D  ' eX  such  that 
•■Ti  n  n  -■       n  ^      n   n 

O  ^(X  ,D  •)  >  Q  ^(X  ,D  *)    i  =  1,2,. ..,L 
Tinn=^nnn 

L 

and  which  maximizes  E  o   (X  ,D  )  over  X  .   The  point  D  '  is  obviously 

.  ,  ^n   n  n        n       ^      n 
1=1 

nondominated  since  otherwise  there  would  be  some  D  eX  with 

n  n 

Q  ^(X  ,D~)  >  Q  ^(X  ,D  •)   for  at  least  one  i 
n   n  n     n   n  n 
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and 


0   (X  ,D  )  >  Q   (X  ,D  •)  for  the  remaining  k  5^  i, 
^n   n  n  =  ^n   n  n 


Then,  of  course, 


EO   (X  ,D  )  >  SO   (X  ,D  •)  which  contradicts  the  choice 
^n   n  n     ^n   n  n 


of  D  ' 
n 


Since  D  '  is  nondominated 
n 


Q  (X  ,D  )  >  Q  (X  ,D  ')  =  >Q  (X  ,D  )  =  Q  (X  ,D  ') 
^n  n  n  =  ^n  n  n      ^n  n  n    ^n  n  n 


V-  D  e  X 
n    n 


Beca\ise  U  is  monotone  increasing,  then  also,  letting  D  =  D  * 

n    n 


Q  (X  ,D  *)  >  Q  (X  ,D  •)  =  >Q  (X  ,D  *)  =  Q  (X  ,D  ') 
^n  n  n   =  ^n  n  n      ^n  n  n     ^n  n '  n 


implies 


U[Q  (X  ,D  *)]  >  U[Q  (X  ,D  •)]=  >U[Q  (X  ,D  *)  ] 
■■^n  n  n   ■"  =   "-^n  n  n  -■     "-^n  n  n   -" 

"■  n  n  n 


so,  D  '  also  solves 
n 


MAX  U[Q^(X  ,D^)]  =  U[Q^(X  ,D^')] 
*■  n  n  n  -"     "■  n  n  n   ^ 


D  eX  , 
n  n 


Thus,  it  is  shown  that  at  least  one  policy  at  which  Ufo  (X  ,D  ) ] 

achieves  its  maximxaa  over  X  is  nondominated 

n 

Q.E.D. 
The  theorem  provides  the  justification  for  solving  for  nondominated 
solutions . 
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Next,  let  us  briefly  explain  the  theory  of  the  dynamic  pro- 
gramming for  the  MCO  problem.   In  dynamic  programming  the  N-variable 
problems  must  be  broken  into  N  one-variable  problems.   So  our  objective 
is  to  decompose  the  problem 

°N'  — '^1 
where  T^(y  =  (g(r^(X^,D^)  ,  Vi(X^_^,D^_^)  ,— 7  (X^,D^)  )  ,  ^  D^,— D,} 
that  is,  T  (X  )  is  the  set  of  all  feasible  values  of 

^^^N^^'V'  Vl^Vl'Vl^'  — '^1^^1'V^  f°^  ^^^  °N'— ^1 
subject  to 

X   .  =  t  (X  ,D^),   n=l,2, ,N 

n-1    n  n  n 

into  N  equivalent  subproblems  each  containing  only  one  state  variable 
and  one  decision  variable.  Then  the  solutions  from  the  subproblems  are 
combined  to  obtain  the  solution  to  the  original  problem.   To  achieve 
this  decomposition,  a  highly  restrictive  assumption  must  be  made  about 
the  function  g. .   That  is,  a  sufficient  condition  for  decomposition  is 
that  g  must  be  a  monotonically  nondecreasing  function  and  another  con- 
dition for  the  decomposition  is  the  separability. 

In  the  case  of  a  single  objective  function,  a  sufficient  con- 
dition for  achieving  the  decomposition  has  been  given  by  Mitten  [33]. 
We  will  prove  that  a  condition  similar  to  Mitten's  is  a  sufficient 
condition  for  the  case  of  multiple  objective  function  in  theorem  [2], 
[3],  [4]. 

To  decompose  the  problem 

°N'  — '^1 
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we  must  achieve  the  crucial  step  of  moving  the  optimization  (here  finding 

all  nondominated  solutions)  with  respect  to  D   , ,D^  inside  the  Nth 

stage  return.   Let  the  multiple  objective  function  be  separable  as 
follows : 

=  9i<^n"S^'V'  92<Vi'Vi'°N-1>'  — '^i"=1'V" 

then 

From  the  definition  of  F  (X  )  it  follows  that 

Vi'— '°i 

where 


Vi^Vi^  =   ^2(Vi^Vi'Vi^'— '^i^^i'V^' ^  Vi'— ^1 

Then  the  decomposition  is  as  follows: 


°N 


where 
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It  is  clear  that  we  can  proceed  further  by  treating  ^ja.i  (^nt_i)  ^^^  then 

F  ^  (X.  J,  —  ,F^(X^)  in  the  same  way  as  F^  (X^  )  .   We  decompose  the 
N-2   N-2       2       2  N   N 

original  problem  into  N  one-stage  initial  state  optimization  problems. 

1.   Fj^CXj^)  =  ^  [S^{X^)] 
°1 
where  S^(X^)  =  {r^(X^,D^)  ,  V- D^} 

n.  F  (X  )  =  sT  ri"  (X  )  1 
n  n      ••  n  n  -• 
D 
n 

where  ?  (X  )  =  {g(r  (X  ,D  )  ,  F  ,  (t  (X  ,D  ) ) )  ,  ¥^  D  } 
n  n        n  n  n    n-1  n  n  n        n 

"here  ?^(X^)  =  (gU^fx^.D^)  ,  Vi"n"S.' V  "  '  *  °n' 

Stating  the  N  problems  more  compactly,  we  have 


F^(X^)  =  ^    [S..(X^)],     n  =  1,2, ,N 

n  n      "•  N  n 
D 
n 

where  ?  (X  )  =  {g(r  (X  ,D  )  ,  F   ,  (t  (X  ,D  ))),¥■  D  } 
n  n     ^   n  n  n    n-1  n  n  n        n 

=  {Q  (X  ,D  )  ,  V^  D  } 
n  n  n      n 


and  Q(X,D)=r(X,D)       n=l 
n  n  n     n  n  n 


=  r  (X  ,D  )  o  F   ,(t  (X  ,D  ))  n  =  2,3, ,N 

n  n  n     n-1  n  n  n 


This  equation  represents  the  usual  recursion  equation  of  dynamic  program- 
ming, but  for  our  application  the  usual  max  or  min  is  replaced  by  the 
operator  fl. 
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We  are  now  in  a  position  to  prove  the  decomposition  in  the  case 
of  multiple  objective  function.   Given  the  separability  and  monotonicity 
of  g.  we  must  prove 

°n-  —  -°i 

where 

^n'V  =   9i<^n'=^N'V'  92'Vl'Vl'Vl''--^l<'=l'°l"'' 
For  the  convenience  of  proving  the  decomposition,  we  transform 

the  multiple  aggregate  function, 

^^^' Vl'— ^1'  ^N'— V  ^°  ^^^'°N'  — '^1^ 
by  using  the  stage  transformation 


X   ,  =  t  (X  ,D  ) 
n-1    n  n  n 


The  stage  multiple  return  is 


r  =  r  (X  ,D  ) . 
n    n  n  n 

From  the  transformation,  it  follows  that  X  depends  only  on  the  decisions 
made  prior  to  stage  n  (D   ., D  )  and  X^,  that  is, 

\   =  ^n+l^Vl'Vl^  =  Vl^V2^V2'V2^'  Vl^ 

=  Vl^\-f2'V2'Vl^  =  Vi^Vs^Vs'Vb^'  V2' 


Vl^ 


=  —  =  ^n+l^\'°N'  — 'Vl>- 
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It  then  follows,  by  combining  the  above  equation  with  the  return 
fionction,  that  the  return  from  stage  n  depends  only  on  the  decisions 

7  =  7  (X  ,D  )  =  7  (t    (X  ,D  , ,D    ),  D  ) 

n    n  n  n     n  n+1  N  N      n+1    n 

=  7  (X^wD„, D  )  . 

n   N  N     n 

The  multiple  aggregate  return  function  R  from  stages  one  through  N  is 
some  function  of  the  individual  stage  returns  written  as 

However,  as  just  explained,  (X^_-,  » /X  )  can  be  eliminated  from  the 

individual  stage  returns  and  consequently  from  the  total  return.   Thus 
an  alternate  expression  for  R  is 


Let  the  multiple  aggregate  return  fionction  be  separable  as 


I         I 


follows 


and  g  be  monotonic  nondecreasing  function  of  g  ¥■  fixed  r 
where 


/    / 
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5i  = 


1 


2 


i  =  1,2 


Let 


where 


^N'  — '^1 


^N^V  =  ^^l^^N^^'V'  ^2^Vl^^'°N'Vl^ 


»    f 


^l^^'^N'Vl'  — 'V^^'  ^°N'  — '°1^ 


T  (X  )  is  the  set  of  all  feasible  values  of 
N   N 


^I^S^^J'V'  52^Vl^Vl'^N'^N-l^'  — ' 
^l^^'^N'  — 'V^^ 


for  all  D^.D^_^,  — ,D^ 


and  fi  C'^M^\r^-l    ^^   ^^®   nondominated  solutions   among  the  values   in 


°N'  — '''l 


the  set  of  T    (x  ) . 


N 


where 


^N^V    =      %^^N^^I'V'  "  tVl^Vl^l^'    ^°N^ 


Vi'— '^ 
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^'^  Vl'Vl'  =  '92<r^.j.<X„,D^,D^.j_),  — ,  r^<X^,D^.  — -D^)) 


we  must  prove 

For  notational  simplification  let 

Note  a  z  e  ^vi^\r^  "^^  nondominated  in  comparison  with  all  other  feasible 
values,  while  awe  ^M^\r^  ^^  nondominated  in  comparison  with  the  values 


enumerated  in  the  evaluation  of  ^    [S  (X  ) ] . 

°N 
4.   Theorem  [2] 

{w}  c_  {z} 

PROOF 

Suppose    a  w  such   that  w   e    {w} 

then  from  w  £    {w}   =  fi    [s"„(X  )  1 

^       "■    N      N 

for  some  D^'.D^.j^*- .O-^* 

Then  it  is   false   that   there   exist  D„,D     ,, ,D,    such  that 

N      N-1  1 
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for  any  D  ., ,D  since  w  is  nondominated. 

But  if  w  ;sf  {z},  this  cannot  be  for  either  of  two  reasons: 

(1)  either  w  is  dominated  in  {z} ,    then  there  exist  D  ,D   , ,D 

such  that 

This  contradicts  the  assumption  w  e  {w} . 

(2)  or  else  w  ft   {z}  can  occur  because  w  was  not  eniomerated  when 
{z}  was  computed, 

but  this  cannot  be  because 


*  °N'  — '^l' 


that  is. 

This  can  be  proved  by  the  following  proof 


Vi'— '^1 


"   .  IVi'Vi'l^tVi'Vi'l 


We   add  rfX^D  )    to  both  sides,    then 
N      N       N 


^%^^N^^'V'        "      [Vl^Vl^l^^  ^  ^^l^^N^^'V 


Vi'— '°i 


Vi^Vi^^^ 


By  monotonicity  of  g  ,  this  equation  is  true . 


And  this  can  be  expressed  as  follows  using  the  definition  of  S  (X  ) , 

N  N 


34 


and  H  [\(\)]   £  ^S^(V^ 


^N 


thus  Q  (s^cy)  c  {s^(v>  £  ^^N^V^ 

So  w  was  enxjmerated  when  {z}  was  computed.   Thus  if  W  £  {w},  we  also 
have  w  e  { z } . 

5 .   Theorem  [3] 

{z}  £  {w} 

PROOF 

Suppose  there  exist  z  such  that  z  £  {z} 

(1)  Then  from 

z  £  {z}  =     n     [T^^X^)] 
^N'  — '^1 

where 


>  I 


■^I'^'^N'  — 'V"'*°N-l'--°l' 


for  some  D  *, D  * 


^  =  ^l^^N^^^'V^'  92(Vl(\'Wl*^'  — '^1%'V'  — '°1*)^^ 


and  for  all  d^,D^_^,  — ,D^. 


35 


The   following        equality  is   false: 

(2)    Also  if  we   fix  X^/D   *,    then 

Vi'— '°i 

where 

Since  otherwise  its  domination  (say  by, 

g2(^N-l(^'V'Vl^'  ^N-2^^'V'  — '^2^'  — ' 

^1^^'V'  — 'V^^^ 
would  imply  domination  of  z  by 

^l^^N^^'V^'  ^2(Vl(^'V'W'  — '^l^^^'V'  — '"^1^^^ 

(by  monotonicity  of  g, )  implying  z  j^    {z}. 
And  now  suppose  z  ^  {w},  this  means  either 

(a)  z  was  not  enumerated  when  {w}  was  computed.   But  this 
cannot  be  by  (2) ,  or 

(b)  z  was  dominated  in  {w},  then  there  is  some  D  , ,D^ 

combination  such  that 
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but  this  contradicts  (1) . 

Thus  if  z  e  {z},  we  also  have  z  e    {w}. 

6.   Theorem  [4] 

{w}  =  {z} 

PROOF 

By  Theorems  [2]   and  [3]. 
From  Theorems  [2],  [3]   [4],  we  have  shown  that  if  g  is  a  monotonically 
nondecreasing  function  of  g  for  every  r  then 

°N'  — '^1  ^N 

where 

Vi'— '"i 

Thus,  the  stage  by  stage  dynamic  programming  recursion  will  generate 
exactly  the  set  of  all  nondominated  solutions  to  the  original  multiple 
criterion  optimization  problem. 


37 


III.   BASIC  COMPUTATION 

A.   GENERAL  SCHEME  (DISCRETE  CASE) 

Computational  aspects  of  dynamic  programming  approach  to  MCO  concern 
the  solution  of  the  multiple  recursion  equation.  In  the  last  chapter  we 
defined  the  multiple  recursion  equation  as  follows. 


F  (X  )  =  Q     [S„(X^)] 
n  n       •■  n  n 

n 


(1) 


where 


S  (X  )  =  {Q  (X  ,D  ) ,  ¥  D  } 
n  n     *n  n  n      n 


Q  (X  ,D  )  =  r  (X  ,D  ) 
''n  n  n     n  n  n 


n  =  1 


(2) 


=  r  (X  D  )  o  F    (t  (X  ,D^)) 
n  n  n     n-l  n  n  n 


(3) 


n  =  2, . . .  ,N 


with 


X   ,  =  t  (X  ,D  ) 
n-l    n  n  n 


First,  we  start  with  n  =  1  (Stage  1)  calculating 


r^(X^,D^)  = 


r^(X^,D^) 


r^(X^,D^) 


r^(X^,D^) 


since  n  =  1.   From  Equation  (2) , 


Ql(X^,D^)  =  r^(X^,D^) . 
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Equation  (1)  is  then  used  to  obtain  F^(X^)  and  D*(X  )  by  using  the  non- 
dominated  method.  These  are  saved  for  future  calculations.  But  Q  and 
r,  are  no  longer  needed.   Since  n  7^  N,  we  increase  n  by  1  and  calculate 

r^Cx^^D^) 

for  n  =  2.   Q^  is  calculated  from  Equation  (3)  by  appropriately  combining 
2 

r^  and  F  .   F^  has  now  served  its  purpose  and  may  be  discarded. 

The  calculations  continue  similarly  for  n  =  3,...,N.   Thus  we  obtain 
F  (X^) ,  the  nondominated  solutions  from  the  N-stage  system,  and  D  (X  ) , 
(n  =  1, . . . ,N) ,  the  optimal  nth-stage  decision  function  of  the  nth-stage 
inputs . 

The  next  step  is  to  determine  the  optimal  inputs  X*  (n  =  1,...,N-1) 
and  the  optimal  decision  D*  (n  =  1,...,N) .   We  have  reached  the  point  in 
the  flow  chart  (see  Fig.  1)  where,  for  the  first  time,  the  answer  to  the 
question  "Does  n  =  N?"   is  "yes." 


We  assvmie  there  is  a  prescribed  value  of  X  denoted  by  X^,  D  is 

it 

obtained  immediately  from  the  stored  fionction  D  (X^)  .   Then  X^  ^  is 
calculated  from  X*    =  t^^^T*'^  *)  .   Since  n  7^  1,  we  compute  D*    from 
X*    and  the  stored  function  D   , (X^_^).   When  the  answer  to  the 
question  "Does  n  =  1?"  is  "yes,"  the  optimal  solution  is  obtained. 


39 


0 


Compute 

r    (X    ,D    ) 
n     n     n 


Q    (X    ,D   ) 
Ti     n     n 

=   r    (X    ,D  ) 
n     n     n 


Solution  V 


Q    (X    ,D    )=r    (X    ,D    )o   F      .  (X         ) 
n     n     n        n     n     n  n-1     n-1 


X        =t    (X    ,D    ) 
n-1     n     n     n 


F    (X    )    =      ^    [S    (X    )] 
n     n  Dj^       n     n 


X*    ,    =t    (X    *,D   *) 
n-1        n     n        n 


1 _ 1 

I  SAVE         F     (X    )  1 

I ?_^ j 

I T 1 

.'  SAVE         D    (X    )  I 

I  n      n  I 


[Figure   1] .      Flowchart  of  General   Computation 
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B.   TABULAR  COMPUTATION 

The  term  tabular  computations  is  used  to  describe  dynamic  programming 

analyses  in  which  the  optimal  decision  functions  and  optimal  return 

fionctions  are  given  in  discrete  f  oinn  in  lists  or  tables .   In  other  words , 

for  each  feasible  value  of  X  ,  n=l,...,N  there  is  an  entry  in  a  table 

containing  values  of  D  *  (X  )  and  F  (X  ) .   (See  Table  1.) 
^  n   n       n  n 

Naturally,  it  is  only  possible  to  give  this  information  in  a  table 

when  there  are  a  finite  number  of  feasible  values  for  X  .   But  tabular 

n 

representation  of  continuous  data  is  possible  if  we  make  discrete 

approximations.   For  example,  suppose  the  feasible  values  of  X  are  the 

integer  numbers  from  1  to  K  .   For  X  =  K  (=1,...,K  ).   D  * (X  =K)  is  the 

n        n  n     n   n 

set  of  all  D  which  are  nondominated  points  of 


Q  (X  =K,D  )  =  r  (X  =K,D  )  o  F   , (t  (X  =K,D  )) 
n  n    n     n  n    n     n-1  n  n    n 


and  F  (X  =K)  is  the  nondominated  solution  value  of  the  above  expression, 
n  n 

At  the  first  stage,  the  feasible  values  of  X^  are  the  integer  numbers 


from  1  to  K  .   Suppose  that  for  each  feasible  value  of  X,  (0,1,2, ... ,K  ), 
n  In 

the  feasible  decision  variable  value  is  D  =  0,1,...,X  .   (See  Table  1.) 

For  each  value  of  X  ,r  (X  ,D  )  is  calculated  when  X  =K.   The  multiple 
return  fionctions  are 


r^(K,0),  r^(K,0), ,  r^(K,0) 

r^(K,l),  r^(K,l),  — ,  r^(K,l) 


r^(K,K),  r^(K,K), ,  r^(K,K) 
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and  D,  is  a  nondominated  point  if  there  does  not  exist  another  D'  such 
that 


rJ^(K,Dp  >  r^(K,D^)  ¥-^ 


i    '     i 
r^  (K,D  )  i-   r-(K,D-)    At  least  one  i 


D^  =  0,1,2, ...,K     D^  ?«  Dj_ 


D'  =  0,1,2, . .  .,K 


that  is 


where 


S^(X^)  =  {Q^(X^,D^)  V  D^} 


The  tables  of  F^  (X  )  and  D^ (X  )  are  saved  for  future  calculations 
(See  Table  2.) 
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Finally,  suppose  that  at  the  last  stage  (N) ,  the  feasible  value  of 

the  state  variable  is  just  one  (K  )  and  the  feasible  values  of  decision 

variable  are  0  through  K^,  and  eveiy  entiry  in  the  table  is  the  same  as 

in  Table  1.   The  calculations  are  basically  done  in  two  parts;  first, 

the  calculation  of  F„(X„)  and  D*(X,),  D*(X.), ,  D*(X  )  and  the  second, 

NN        1122         NN 

the  tracing  of  the  nondominated  polxcy  D  , ,D  . 

At  the  last  stage  we  get  X  *  =  K  and  D   then  by  transformation 
^      ^    N     n      N 

X    =  t  (X  ,D  *)  we  can  compute  X*_  and  by  the  summary  table  of 

Stage  N-1  (we  saved  each  ^^-l^^-l^'  ^^'^' '^'°N-1  ^\-l^  ^  *   "®  ^^ 

compute  D*  ^ (X^  ) .   We  will  repeat  this  until  n=l;  then  at  least  we  will 

9^*^  °N'  °N-1'  — '°1* 


C.   AN  EXAMPLE:   ONE  STATE  VARIABLE  AND  IRREGULAR  RETURNS  AND  TWO 
OBJECTIVE  FUNCTION 


This  example  shows  how  tabular  computations  are  organized  and 
provides  some  clues  to  the  niomber  of  computations  involved  when  the 
recursion  equations  are  solved  by  enumeration. 

Example  Problem. 

Five  OA  students  meet  to  work  the  homework  problems  in  two  courses, 
D.P.  and  Stochastic  Models;  both  have  three  problems.   They  agree  to 
split  into  three  teams... each  team  assigned  one  D.P.  problem  and  one 
stochastic  problem.   The  probability  of  success  on  a  problem  depends  on 
the  number  of  students  working  on  it.   (No  student  can  work  on  more  than 
one  team.   Assume  all  students'  capabilities  are  equal.) 
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D.P. 


Nvunber 
of  Students 

Probability 

Problem  1 

Problem  2 

Problem  3 

0 

.0 

.0 

.0 

1 

.1 

.6 

.3 

2 

.2 

.8 

.5 

3 

.3 

.9 

.6 

4 

.4 

.9 

.7 

5 

.5 

.9 

.7 

Stochastic  Project 


Niamber 
of  students 

Probability 

Problem  1 

Problem  2 

Problem  3 

0 

.0 

.0 

.0 

1 

.7 

.2 

.1 

2 

.8 

.3 

.4 

3 

.9 

.4 

.5 

4 

.9 

.5 

.6 

5 

.9 

.6 

.7 

How  many  students  should  be  on  each  team  to  maximize  the  joint  proba- 
bility that  all  problems  in  D.P.  are  solved  and  the  joint  probability 
that  all  in  Stochastic  are  solved?   (At  least  one  student  must  be  on 
each  team.)   The  formulation  of  problem. 

3    1 
Maximize   tt  P   (D  ) 
n   n 
n=l 

n=l 
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Subject  to    E  D   <  5 
n=l 


D   >  0,  D  =  integer 
n  —     n 


where 


P  =  D  P  probability 

2 
P  =  stochastic  probability 

n  =  1,2,3  (niomber  of  problems) 


Multiple  recursion  equations 


F^(X^)  =  fi  [S-l(X^)],  where  S^(X^)  =  {P^(X^,D^),  ¥■  D^} 
D, 


F-(X_)  =  fi  [S,(X-)],  where  S_  (X  )  =  {Q  (X  ,D  )  ,  ¥•  D  } 


2   2'     '  '^2'  2 
°2 


2^2' 


'2'  2'  2' 


Q^iX^)    =P2(X2,D2)  OF^CX^-D^) 


F^CX^)  =  fi  [S^CX^)],  where  S^CX^)  =  {Q3(X2,D2),  ^  D^l 
°3  _        _  _ 


Computations 


Stage   1 


F=  [P\  p2f 


^1 

^1 

^i<v 

P^D^) 

^l^V 

1 

1 

.1 

.7 

.1           .7 

2 

1 

.1 

.7 

2 

.2 

.8 

.2           .8 

3 

1 

.1 

.7 

2 

.2 

.8 

3 

.3 

.9 

.3           .9 
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Summary    Table    1 

=^1 

F^(X^) 

D^MX^) 

1 

.1              .7 

1 

2 

.2             .8 

2 

3 

.3              .9 

3 

Stage    2 


^2 

°2 

t^{X^,D^) 

P^(D2) 

P^^D^) 

^1^^2^^2' 

D^)) 

22(^2 

.D^) 

F^CX^) 

2 

1 

1 

.6 

.2 

.1 

7 

.06 

.14 

.06 

.14 

3 

1 

2 

.6 

.2 

.2 

.8 

.12 

.16 

.12 

.16 

2 

1 

.8 

.3 

.1 

.7 

.08 

.21 

.08 

.21 

4 

1 

3 

.6 

.2 

.3 

.9 

.18 

.18 

.18 

.18 

2 

2 

.8 

.3 

.2 

.8 

.16 

.24 

.16 

.24 

3 

1 

.9 

.4 

.1 

.7 

.09 

.28 

.09 

.28 

Summary    Table     2 

=^2 

F^tX^) 

DjMX^) 

2 

.06           .14 

1 

3 

.12           .16 
.08           .21 

1 

2 

4 

.18           .18 
.16           .24 
.09           .28 

1 

2 
3 

Stage    3 


^3 

°3 

t3(X3,D3) 

P^(D3) 

"3 '"3' 

F2(t3CX3,D3)) 

^s^^S'^S^ 

F3(X3) 

5 

1 

4 

.3 

.1 

.18           .18 
.16           .24 
.09           .28 

0.054   0.018 
0.048   0.024 
0.027    0.028 

2 

3 

.5 

.4 

.12           .16 
.08           .21 

0.060   0.064 
0.040   0.084 

0.060   0.064 
0.040   0.084 

3 

2 

.6 

.5 

.6              .14 

0.036   0.070 
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By  back  tracking  we  find  the  nondominated  solution  to  contain  two  parts 

imply: 

(1)  D^  =  2  D.P.  =  0.06 

D-  =  1  Stochastic  =  0.064 

(2)  D   =  1  D.P.  =  0.04 

D  =  2  Stochastic  =  0.084 
D3  =  2 
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stochastic 


0.08-. 


0.07-. 


0.06-. 


0.05- 


0.04-' 


0.03-. 


0.02-- 


(2,1,2) 


(3,1,1) 


0.01.. 


4- 


— I — 

0.02 


■f- 


■+■ 


■+■ 


D.P. 


0.01 


0.03 


0.04 


0.05 


0.06 


Figure  2.    Nondominated  solutions  in  two-dimensional 
criteria  space. 


D.   THE  CONTINUOUS  VARIABLE  CASE 


Let  us  consider  the  problems  involved  in  solving  MCO  problem  when 
we  no  longer  require  that  the  D.  be  integer.  We  shall  then  also  drop 
the  restriction  that  a.  ,  K  be  integers-   That  is 

where 

^n'V  =  '9(r„'>^'V'  Vl'Vl'Vl''  — '^l^l-V'- 
In  carrying  out  the  optimization  we  consider  only  non-negative  D .  which 
satisfy 

n 

Z  a.D.  <  K  . 

j=i  :3  :  -  n 


Precisely  as  before. 


F^(X^)    =  Q  [S^(X^)] 


o<D- <K  /a, 
—  i —  n      1 


where 


S^(X^)    =    {r^(X^,D^) ,    V  D^} 


Note  that  D-  varies  up  to  K  /a.  not  [k  /a. 1 .   The  same  recurrence 
1  n  1     *■  n  l"* 

relations  follow  that  applied  to  the  case  where  the  D.  were  to  be  integer. 


That  is 


F  X   =    fi      [S   X 
n  n      ^  .,  /    n  n  -■ 
o<D  <K  /a 
—  n—  n  n 


51 


where 

S"  (X  )  =  {qL  (X  ,D  )  ,  ^  D  }, 
n  n     *n  n  n      n 

Q  (X  ,D  )  =  r  (X  ,D  )  o  F   , (X  -a  D  ) . 
^n  n  n     n  n  n     n-1  n  n  n 

In  order  to  apply  the  dynamic  programming  computation  using  the 
tabular  form,  the  decision  variables  must  be  discrete.   So  first  we 
must  decide  on  a  discrete  grid  to  approximate  the  continuous  variables. 
If  the  grid  size  is  A,  the  length  of  interval  (L)  then  the  number  of 
points  (p)  on  which  the  function  is  defined  is 

L 

Thus  for  the  fixed  L,  p  is  inversely  proportional  to  A.   If  p  is  too 

large,  then  the  D.P.  is  almost  impossible  to  calculate,  we  must  reduce 

the  size  by  making  the  grid  coarser.   That  is,  by  changing  the  grid  from 

A  to  a  larger  value  6  there  is  a  corresponding  decrease  in  p. 

To  see  how  the  computations  decrease  as  the  grid  increases,  suppose 

we  want  the  optimization  of  Q  (X  ,D  )  for  fixed  X  ,  and  o<D  <20,000  on 

^n  n  n  n       —  n— 

the  grid  d  =  0.1, ,20,000.   If  we  start  initial  grid,  there  are 

20,001  points  at  which  the  fxmction  is  to  be  evaluated. 

By  increasing  the  initial  grid  size  to  d  =  0,  100,  200, ,20,000 

and  evaluating  the  f\anction  at  201  points,  the  interval  of  optimality  is 
reduced  to  a  length  of  200.   Then  another  201  evaluations  of  the  function 
will  yield  the  integer  solution.   But  the  number  of  computations  can  be 
reduced  still  further  by  starting  with  a  very  coarse  grid  d  =  0,  1000, 

2000, ,  20,000.   Then  the  interval  of  optimality  is  reduced  to  a  length 

of  20.   Then  another  21  evaluations  of  the  function  will  yield  the  inte- 
ger solution. 
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Assume  that  the  function  F   , (X   , )  is  known,  then  Q  is  defined 

n-1  n-1  n 

over  a  continuum  of  values  D  in  the  interval  o<D  <K  /a  . 

n  —  n —  n  n    • 


If  Q. 


n 


1  2 

is  plotted  (as  Q   (X  ,D  )  vs  Q   (X  ,D  ) 
^  *nnn     *nnn 

(at  fixed  X  ) 
n 


it  might  look  like  a  space  or  a  curve.   (See  Figure  1.) 


[Figure  1] 

And  the  nondominated  solution  set  is  the  thick  line.   We  would  like  to 

determine  the  set  of  values  D  (X  )  at  which  Q  assumes  its  nondominated 

n  n  ^n 

solution.   To  reduce  as  much  as  possible  the  computational  effort 

reqiiired,  one  might  do  a  rough  initial  search  using  a  coarse  grid  of 

D  ,  (D  =  0,5,2  5, D  ).   (See  Figure  2.) 

n    n  n 
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n 
[Figure  2] 

From  this  coarse  grid  of  decision  variables  an  approximate  nondominated 
policy  would  be  found  from  Figure  2,  points  A  and  E.   Then  we  can 
conclude  that 

d*-6<   dA*n  <  d5*  +  5 


that  is. 


0  _<  dA*n  <_  6 
36  <  dA*n  <  55, 


A  finer  grid  would  be  used  in  the  neighborhood  of  the  nondominated 

point  to  determine  more  nondominated  values  or  to  locate  them  more 

accurately.   The  procedure  of  using  a  finer  grid  might  be  repeated 

several  times  to  obtain  the  desired  number  of  nondominated  points  in 

D  (X  )  . 
n  n 


M  '  "  '       ''  ■    I  M  '  '  '  I  I  I  M 

A    B    C    D    E    F 


[Figure  3] 


54 


Consider  next  the  problems  which  arise  in  the  tabulation  of  the 

F  (X  ) .   A  difficulty  is  encountered  here  because  F  (X  )  is  defined  for 
n  n  n  n 

all  X  ,  o<X  <K  .   Furthermore,  examination  of  the  search  procedure 
n   —  n—  n 

introduced  above  shows  that  F  (X  )  may  actually  be  needed  for  any  X  in 

n  n  n 

this  interval,  due  to  the  fact  that  when  finer  and  finer  grids  are  used 

to  determine  F  (X  )  for  a  given  S  ,  F   , (X   .)  may  be  required  for  a 
n  n  n   n-i  n-± 

very  large  number  of  different  arguments.   It  would  usually  be  impossible 

to  tabulate  the  F  (X  )  for  all  possible  X  that  might  be  required  with 

the  search  procedure  described  above.   Clearly,  if  this  cannot  be  done, 

we  must  tabulate  the  F  (X  )  for  a  smaller  number  of  X  . 

n  n  n 

In  dynamic  programming  the  main  benefit  of  a  coarse  grid  on  the 
decision  variable  is  that  it  produces  a  corresponding  coarse  grid  on 
state  variables . 

In  the  relation 


X   ,  =  t  (X  ,D  ) 
n-1    n  n  n 

the  number  of  feasible  values  of  X   -,  is  roughly  proportional  to  the 

number  of  feasible  values  of  D  .   By  reducing  the  number  of  feasible 

values  of  X  , ,  we  evaluate  F  , (X  ,)  fewer  times  and  the  computational 
n-1  n-1   n-1  ^ 

procedure  is  sames  as  discrete  case.   The  successive  grids  used  in  a 
dynamic  programming  analysis  depend  on  the  number  of  values  tolerable  for 
the  state  variables  and  the  accuracy  required. 


E.   ODMPUTATIONAL  EFFICIENCY  OF  THE  METHOD 

Let  us  now  compare  the  computational  efficiency  of  the  dynamic  pro- 
gramming approach  to  solving  the  MCO  problem  with  that  of  simply 
enumerating  all  possible  sets  of  non-negative  integers  D.  which  satisfy 
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the  constraint 


N 

E  a.D.<Kn. 


For  illustrative  purposes 


N 
let  a.  =  1  then  Z  a.D.<K  become 
^  j=l  ^  ^-  ^ 


N 

Z   D.<Kn  (1) 

3=1  '- 


and  only  one  constraint  and  one  variable. 


1.   Direct  Enumeration 


In  the  case  of  N  stages  and  L  objective  functions  and  the 
constraint  is  (1)  then  the  number  of  evaluations  of  objective  functions 


xs 

Kn 


K=0     ^ 


and  the  number  of  comparison  for  choosing  nondominated  solutions  is 

_   K  K 


K=0 


12  L 

This  shows   the  nimiber  of  comparison  of  vectors    (f    ,    f    , ,f  ) 


2.      P.P.   Approach 

At  each  stage,  there  are  (Kn+1)  x  values  and  for  each  value  of  x 
there  are  (x+1)  D  values.   The  number  of  evaluation  of  objective  functions 
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L   E   (x+1)  =L((Kn+l)  +  Kn(Knj:lI 


at  one  stage  is 

Kn 

E 
x=0 

All  (N-1)  stages  have  the  same  number  of  enumerations 


L  X  (N-1)  [Kn+D  +  SUfEill] 


and  last  stage  has  just  (Kn+1)  x  L  computations.   Thus 


[(N-l)((Kn+l)  +  ^^^'■^^)    +  (Kn+1)]  X  L 


and  the  number  of  comparisons  for  choosing  the  nondominated  solutions  is 

at  each  stage 

Kn 

S   x(x+l) 


x=l   2 

and  since  all  (N-1)  stages  are  the  same,  the  total  number  is 
(N-1)   E   '''^•'" 


x=l   2 


At  the  last  stage,  the  number  of  comparisons  is 

thus  the  total  number  in  N-1  stage  is 

/M  IX   V  x(x+l)  ,  Kn(Kn+l) 
(N-1;   2-  — + 


1    2         2      * 

x=l 

Therefore   total  nromber  of  comparisons    for  all   stages   is 


(N-1)  [^(Kn+1)  (2Kn+l)    +  KfliKn+lI  ^  Kn(Kn+l) 


-12 
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For    example     (2    objective    functions) 


DIRECT    ENUMERATION 


^„ 

N\ 

10 

50 

100 

1000 

3 

572 

4.7x10^ 

3.5x10^ 

8 

3.3x10 

5 

6x10"^ 

6.9x10^ 

1.9x10^ 

1.7x10''"^ 

10 

3.7x10^ 

l.SxlO''""'" 

9. 2x10"'"^ 

5.8x10^^ 

DYNAMIC    PROGRAMMING 


Vn 

.X 

10 

50 

100 

1000 

3 

286 

5.4x10 

4 
2 . 1x10 

2x10^ 

5 

550 

4 
1x10 

4.1x10^ 

6 
4x10 

10 

1210 

2.  4x10  "^ 

9.2x10^ 

6 
9x10 

Notice   that   the   direct  enumerations   grow  exponentially  and  the   dynamic 
programming  approach   increases   linearly  with  N. 
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IV.   CONCLUSIONS 

We  have  discussed  the  concept  of  the  nondominated  solutions  and  have 
shown  how  the  concept  of  the  dynamic  programming  can  be  applied  to 
determine  the  nondominated  point  set. 

We  have  also  discussed  the  sufficient  conditions  for  the  decomposi- 
tion of  multiple  criteria  functions .   Problems  involving  both  the 
discrete  and  continuous  case  can  be  handled. 
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